This Page Is Inserted by IFW Operations 
and is not a part of the Official Record 



BEST AVAILABLE IMAGES 



Defective images within this document are accurate representations of 
the original documents submitted by the applicant. 

Defects in the images may include (but are not limited to): 



BLACK BORDERS 

TEXT CUT OFF AT TOP, BOTTOM OR SIDES 
FADED TEXT 
ILLEGIBLE TEXT 
SKEWED/SLANTED IMAGES 
COLORED PHOTOS 

BLACK OR VERY BLACK AND WHITE DARK PHOTOS 
GRAY SCALE DOCUMENTS 



IMAGES ARE BEST AVAILABLE COPY. 



As rescanning documents will not correct images, 
please do not report the images to the 
Image Problem Mailbox. 



REMARKS 

This submission is in response to the Office Action dated March 18, 2003. 
Claims 1 , 3, 4 and 6-25 are pending. 

Claims 1 , 3, 4 and 6-25 stand rejected over the combination of Barry, U.S. 
4,093,235 in view of Moore, U.S. 3,642,281 . 

Main claim 1 and other claims have been amended to clarify the language. 
Claim 1 also has been amended to set forth that the gripping wing spaced from the end of the 
elongated parallelepiped body of the multi-token is within the outer periphery of the body. 

As set forth in claim 1 , the multi-token has the following features: 

a. it is an elongated parallelepiped body. The Examiner's reference to a 
pyramid shape (Action, page 2, line 14) is therefore irrelevant. The elongated parallelepiped 
feature permits easy stacking of the multi-tokens. The tokens of the (applicant's) prior Barry 
patent are in the physical form of an object and cannot be stacked; 

b. a visual representation of the various modes of transportation (air, boat, 
auto) are placed on at least three of the elongated flat sides of the token body. This makes 
production of the game less expensive since different shapes of tokens are not required. The 
transportation mode can be painted on a side of the parallelepiped body or a decal or paper 
placed thereon; 

c. there is a gripping wing at each of the ends of the elongated 
parallelepiped body. The gripping wing is spaced from the end of the parallelepiped body 
and leaves a recess . The purpose of the gripping wing is to improve the tactile feel of the 
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token in two respects. First, a fingernail can be inserted into the recess to move the token. 
Second, the flesh of the fingertip that falls into the recess between the wing and the body and 
encounters the ridge surface and the fingernail that falls into the recess produces a different 
force pattern on the fingertip and a firmer grip of the token. Reference is made to the two 
articles by Stephen Mascara and H. Harry Asada entitled Understanding of Fingernail-Bone 
Interaction and Fingertip Hemodynamics for Fingernail Sensor Design and Finger Posture 
and Shear Force Measurement using Fingernail Sensor: Initial Experimentation, attached 
as Exhibits A and B. These demonstrate, for example, see Section 2.2 of Exhibit B, that any 
flexing or bending of the fingertip produces a different force pattern. This enhances the tactile 
"feel" of the fingertip engaging the end of the token and provides a better gripping surface. 
Also, the fingernails can be inserted into the recess between the token body and the gripping 
wing to manipulate the token. 

The claim also recites that the gripping wing is within the outer periphery of the 
body. Therefore, the gripping wings do not interfere with the stacking of the multi-token. 

d . the ends of the multi-token each have a visual indication of the movement 
status of the tokens, i.e., the GO-NO GO indication as set forth in claims 22-23 and 26-27. 
This feature interacts with the gameboard. 

To summarize, the above features set forth in claim 1 have the following 

advantages: 

a. better functionality of the tokens since even with the gripping wings they 
can be stacked; 
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b. economy of manufacture 

c. improved tactile sensing of the multi-token provided by the gripping 

wings; 

d. increased visibility of playing status by the indications on the outer 
surface of the gripping wings. 

Features a., b. and d. also interact with parts of the game that is displayed onthegameboard. 

Turning now to the rejection, the Examiner relies on applicant's prior patent for 
teaching of the game and its layout. Applicant does not dispute this. Applicant contends that 
the invention is in the combination of the game and the novel multi-token. 

In comparing the subject matter of the present invention with applicant's prior 
patent, it is clear that the multi-token provides all of the advantages discussed above. The 
Examiner recognizes that applicant's prior patent does not teach the novel multi-token and 
relies on Moore for this feature. 

There are substantial differences between the playing pieces shown in the 
Moore patent as compared to the multi-tokens of the subject invention as set forth in claim 1 . 
For example, claim 1 sets forth that the complete multi-token is an elongated parallelepiped, 
while Moore's game pieces are cubic shaped. A cube is not an elongated parallelepiped 
body. A cube has equal sides and not the elongated sides of the elongated parallelepiped. 
It is true that because of the cubic shape Moore's playing pieces can be stacked. However, 
it is easier to stack applicant's tokens because the sides are longer. 

Also, when considering the use of Moore's playing pieces as compared to the 
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multi-tokens of the subject invention, there are considerable differences. That is, Moore's 
playing pieces are used in a capture-type of game, like checkers (see column 2, lines 37-67). 
There does not appear to be any interaction between any of the indicia on Moore's playing 
pieces and the parts of the gameboard. In the subject invention the mode of transportation 
indicia on the sides of the playing pieces correspond to parts of the gameboard. For 
example, an airplane indicia on a multi-token matches up with a corresponding area of the 
gameboard having this indicia. In the Moore patent, even referring to Fig. 21 where a boat, 
airplane and auto are shown on the gameboard, there appears to be no such playing 
piece/gameboard interaction. Reference is made to the description of Moore's Fig. 21 at 
column 5, lines 20-55. 

The Examiner states at the top of page 3 of the Action that the applicant does 
not disclose why it is critical for the multi-tokens to be of an elongated parallelepiped shape 
and, because of this, he concludes that this shape would be a matter of obvious design 
choice. Applicant urges that "criticality" is not the issue present here. What is present is the 
question of what advantages the elongated parallelepiped body produces. It is settled law that 
the Examiner must consider arguments and additional evidence submitted relative to 
advantages of structure that is disclosed in the application. Reference is made to MPEP 
j 716.02(f). 

The advantages of the elongated parallelepiped body as compared to a cube 
is that it is much easier to stack the former and that the stack will be more stable since there 
is more surface area in contact. Also, there is more surface area on the flat sides on which 
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to place indicia. 

Also important is the claimed feature of the gripping wings at the ends of the 
multi-token body. The Examiner recognizes that neither Barry nor Moore discloses such 
gripping wings. As discussed above, the advantages of the gripping wing structure relates 
to increasing the tactile utility of the multi-token. This is particularly useful and desirable for 
people having infirmities such as arthritis, and other neurological impairments. 

The Examiner summarily dismisses the important gripping wing feature by 
concluding "that a person having ordinary skill in the art would have expected the sides of 
Moor to functionally perform the same, which is to provide sides for moving the game piece". 
While it is true that Moore's pieces are moved by gripping the sides, the patent does not teach 
or even remotely suggest the claimed gripping wings that leave a recess between the ends 
of the elongated parallelepiped body which provide the increased tactile performance as 
shown by the submitted Exhibits A and B. 

Main claim 1 sets forth a novel and advantageous combination that patentably 
distinguishes over the cited art. There is no logical basis to combine the playing pieces of 
Moore with the gameboard of the Barry patent since there is no teaching in Moore of 
interaction between the indicia on such playing pieces and parts of the gameboard as used 
in the present invention. Even if the combination is improperly made, the structure of claim 1 
is not met for the various reasons given above. The novel design of the multi-tokens of the 
invention, including the gripping wings, provide a number of advantages. Accordingly, claim 
1 is clearly patentable and should b allowed. 
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The other claims depend, in one way or another, from claim 1 . In view of the 
allowability of claim 1 , these claims are also patentable and should be allowed. 

The other active claims depend, in one way or another, from claim 1 and also 
should be allowed. 

The amendment should be entered since it clearly places the application in 
condition for allowance. 

In view of the above amendments and remarks, it is respectfully requested that 
the application be reconsidered and that all pending claims be allowed and the case passed 
to issue. 



resolved through either a Supplemental Response or an Examiner's Amendment, the 
Examiner is respectfully requested to contact the undersigned at the telephone number 
indicated below. 



If there are any other issues remaining which the Examiner believes could be 



Prompt and favorable action is requested. 




SJ*eter Lpdwig 
Reg. No. 25^351 
Attorney for Applicants 



DARBY & DARBY, P.C. 
Post Office Box 5257 
New York, NY 10150-5257 
Phone (212)527-7700 
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Abstract 

When the human fingertip is pressed against a surface 
or bent, the hemodynamic state of the fingertip is altered 
due to mechanical interactions between the fingernail and 
bane. Normal force,, shear force, and finger 
extension/flexion all result in different patterns of blood 
volume beneath the fingernail. This phenomenon has been 
exploited in order to detect finger forces arid finger 
posture by creating a photoplethysmograph "fingernail 
sensor, " which measures the two-dimensional pattern of 
blood volume beneath the fingernail. In this paper, the 
anatomical structure of the fingertip is investigated in 
order to understand the various ways in which the bone 
and nail interact to alter the hemodynamic state of the 
fingertip. A qualitative nail-bone interaction model is 
created and used to explain the different blood volume 
patterns that result from each stimulus. The model is 
verified using experimental data from the fingernail 
sensor. The impact of this study on potential performance 
and application of the fingernail sensors is discussed, 

1. Introduction 

Fingertip forces play ah increasing role in the fields of 
robotics, medicine, and virtual reality [1]. They act as bi- 
directional feedback between human and environment, 
either mechanical or virtual. Forces applied by a machine 
or virtual tool are fed back and presented to the human, 
while forces applied by the human are measured and fed 
back to the machine of Virtual environment. Both 
application and measurement of fingerpad forces are 
required, and understanding the mechanics and dynamics 
of the human fingerpad is important for both. 

Several researchers have investigated the mechanics 
and dynamics of the human fingerpad [2]-[5]. Resulting 
analyses lead to a better understanding of human grasping 
and manipulation, characterizations of the human haptic 
sense, ergonomic design criteria [2], and performance 
criteria for haptic feedback devices [6]. However only a 
few studies have taken into account the role of the 
fingernail in fingerpad behavior [7], It is well documented 



in medical literature that the fingernail plays an important 
role in human grasping and fine manipulation [8 J, [9 J. 

In addition to applying forces, the fingernail has 
recently been discovered to be useful for measurement of 
forces, When forces are applied to the fingerpad, 
interaction between the fingernafl,~bone, and tissue alters 
the hemodynamic state of the finger, creating various 
patterns of blood volume in the capillaries beneath the 
fingernail. In previous works, photoplethysmograph 
fingernail sensors have been designed which optically 
measure the two-dimension pattern of blood volume 
beneath the fingernail [10]. These patterns can then be 
used to predict the fingerpad forces. Normal forces, shear 
forces, and changes in finger posture have all been shown 
to result in different blood volume patterns. 

In order to better design such fingernail force sensors, 
it is important to understand the sensing mechanism, 
including the mechanics of the fingernail-bone interaction 
and hs effect on blood volume. In previous research, the 
mechanism behind, the hemodynamic response to normal 
force has been quantitatively modeled, but the response to 
sheaf force and finger bending were not understood [11]. 

In this paper, a unified qualitative model will be 
created that will explain the mechanism behind the blood 
volume patterns, for normal force, shear forces, and finger 
posture. First, the observable fingernail color patterns that 
are representative of blood volume are described Next, 
relevant structural and vascular anatomy of the fingertip is 
analyzed and used to create the qualitative nail-bone 
interaction model, Experimental data from the fingernail 
sensor is then used to verify the blood volume patterns 
predicted by the model. Finally, the impact of this study 
on potential performance and application of the fingernail 
sensors is discussed. 

2. Fingernail Color Patterns 

As the human fingertip is pressed down on a surface 
with increasing force, the blood flow through the fingertip 
is affected, and a sequence of color changes is observed 
through the fingernail. In fact, the color change is 
characteristically non-uniform across the nail, resulting in 
distinct patterns of color change for different types of 
forces. 



2.1. Normal Touch Force 

Figure 1 shows the typical sequence of noticeable color 
changes with increasing normal force. As the touch force 
is first increased, the veins in the fingertip are collapsed, 
causing blood to pool up m the capillaries beneath the 
nail, resulting in the reddening effect. As the force 
continues to increase, the force propagates around the 
bone, collapsing the capillaries at the tip of the nail bed, 
resulting in a white zone at the tip of the nail. 

Figure 2 shows an example of the effect of normal 
touch force on the fingernail. The pictures on the left 
show the fingernail coloration under ordinary conditions 
with no force. The pictures on the right show the 
coloration with normal force. Just as in Figure 1, the area 
around the bone is whitened, while the area above the 
bone is reddened. Digital filtering techniques are used to 
improve the contrast between red and white zones. In the 
figures on top, the contrast has been multiplied by five, 
and in the figures on the bottom, an intensity threshold has 
been applied. 




e.g. force < 0.3 N e.g. 0.3 N < force < I N 



diminished 




e.g. t N < force < 4 N e.g. force > 4 N 

Figure 1. Fingernail colors due to touching. 




Figure 2. Visible effect of normal touch. 



2.2. Change in Posture 

Normal touch force is not the only action that results in 
a change in fingernail color. When the posture of the 
finger is altered, i.e. the joints of the finger are bent or 
extended, the color of the finger changes as shown in 
Figure 3. When the finger is extended, a tension is set up 
in the tissues of the nail bed that collapses the capillaries. 
When the finger is bent, that tension is relieved and the 
capillaries fill with blood again. The color changes shown 
in Figure 3 are concentrated near the center of the nail, 
whereas the color changes due to normal touching occur 
particularly towards the tip of the nail. Therefore it should 
be possible to distinguish between a touching action and a 
change in finger posture based on observable changes in 
fingernail color patterns. 

Figure 4 demonstrates the visible effect of finger 
extension on the fingernail When the finger is extended, 
whitening occurs between the bone and the fingernail, 
especially at the distal end of the bone. When the finger is 
flexed, the whitening disappears and the entire nail 
reddens. 




Figure 3. Fingernail colors, due to bending. 




Figure 4. Visible effect of extension. 



2.3. Shear Force 

When shear forces are applied to the pahnar surface of 
the fingertip, yet another set of color patterns result, as 
shown m Figure- 5. If the shear force ts applied 
longitudinally, a tension in the tissues of the nail bed is set 
up, resulting in either a broad whitening effect over the 
center of the nail or a white band at the tip of the nail, 



depending on the direction. If the shear force is applied 
laterally, tension in the nail bed creates whitening zones 
that are asymmetrical. 




Figure 5. Fingernail colors due to shear. 



Figure 6 depicts the visible effect of lateral shear force 
on the fingernail. In the picture on the left* the shear force 
is applied from top to bottom* while on the right* the shear 
force is applied from bottom to top. In both cases, the 
fingernail whitens around the bone on the far side of the 
nail toward which the shear vector is pointing, as well as 
over the bone toward the near side of the shear vector. 




Figure 6. Visible effect of lateral shear force. 



Finally, Figure 7 shows the visible effects of 
longitudinal shear force. The picture on the left depicts 
shear applied to the finger from left to right (positive by 
our convention). In this case we see whitening over the 
bone, but reddening at the tip. The picture on the right 
depicts shear applied to the finger from right to left 
(negative by our convention). In this case, we see 
whitening around the bone, which is not significantly 
different from the patterns caused by normal force alone. 

Since the patterns for lateral shear force are distinctly 
asymmetrical, it should be easy to distinguish from normal 



touching and bending. However, longitudinal shear forces 
may present a greater challenge to distinguish. When the 
longitudinal shear is applied inward (top right of figure), 
the whitening zone is similar to that of bending, but 
extends all the way to the lateral edges of the nail. When 
the longitudinal shear force is applied outward (top left of 
figure), the whitening zone is almost indistinguishable 
from that of normal touching, making this perhaps the 
most challenging force to measure. However, there may 
be subtle variations in blood volume that are more visible 
to optoelectronic sensors than to the naked eye. 




Figure 7. Visible effe<^ of longitudinal shear. 



3. Anatomy of Fingertip and Nail Bed 

In order to explain the mechanism behind the changes 
in color of the fingernail, it is necessary to thoroughly 
understand the relevant anatomy and physiology of the 
fingertip. First, the anatomical structure and function of 
the fingertip and fingernail is. investigated. Secondly, the 
blood flow in the fingertip and fingernail bed will be 
investigated 

3.1. Structure of the Fingertip and Fingernail 

Details of the anatomical structure of the fingertip can 
be found in several references such as [&}, [9}, [12}. The 
top portion of Figure 8 shows a saggital cross-section of 
the fingertip, The bone of the distal phalanx is surrounded 
beneath by the soft deformable tissue of the pulp and 
above by the nail unit The nail plate is attached to the 
bone by the anterior ligament (AL), the posterior ligament 
(PL)» and the bed mesenchyme (BM), the latter having an 
almost ligamentary or tendon-like character [8]. This 
anchoring serves to maintain the positional relationships 
and distances between the matrix, bed, hyponychium, and 
bone, which are critical for nail health and functionality. 
The nail plate is generated by the matrix, grows up and 
emerges out from under the proximal nail fold at the 



eponychium, curves over the nail bed, and separates from 
the fingertip at the hyponychium. It is transparent and 
colorless, acting as a window into the nail bed. 




Figure 8. Structural anatomy of the fingertip. 



Figure 8 also shows the actuation mechanism of the 
fingertip. Movement of the finger and forces at the 
fingertip are effected by means of a network of flexor arid 
extensor tendons, [91. The extensor tendons pull the finger 
up into an extended position, while the flexor tendons pull 
the finger down into a bent or flexed position. 

3.2. Blood Supply of the Fingertip and Nail Bed 

According to [8], after application of pressure to the 
pulp, this middle nail bed turns pinker while the distal bed 
becomes whiter, which supports the role of the bed 
vasculature as the main source of the pink color of the 
bed. 

A variety of details on the vascular anatomy of the 
fingertip can be found in several sources such as 
[8]»[9],[I3J-[I6]. The nail bed is richly vascularized with 
blood flowing from the digital arteries into a network of 
arterioles, through capillary loops just under the surface, 
and back out through the venules to the digital veins. 
Figure 9 shows a diagram of the principal arteries and 
veins. The main digital arteries divide into a network of 
smaller arteries that run principally above the bone, which 
is connected to the fingernail via a strong matrix of 
collagen and elastic fibers, as described in the previous 
section. As a result, the arteries underneath the nail are 
protected from touch pressure, allowing uninterrupted 
supply of blood to the capillaries under the nail. However, 
the flow of blood out of the fingertip relies largely on the 
lateral ramifications of the digital veins shown in the 
figure [16]. In addition, the veins are generally larger and 
more compliant than the arteries, leaving them susceptible 
to collapse by touch pressure. As a result, when touch 
pressure is applied to the fingertip, the veins are 



collapsed, causing blood to pool up in the capillaries 
underneath the nail. 




veins 



arteries 

TRANSVERSE VIEW DORSAL VIEW 
Figure 9: Vascular anatomy of the fingertip. 

The capillaries run longitudinally under the nail bed 
and are twice as long and twice as numerous as those in 
the pulp on the palmar side of the fingertip [13}. Under 
normal conditions, the blood in the capillaries is rich in 
oxygen and therefore red. Thus the blood that pools up in 
the capillaries of the nail bed is highly visible and is 
responsible for the reddening effect described earlier. 
However the capillaries under the nail at the tip of the 
finger are not protected by the bone. Thus touch pressure 
can propagate around the tip of the bone, causing these 
capillaries to collapse and pushing all of the blood out of 
them. This results in the whitening effect described 
earlier. 

A final relevant detail is that the arteries are tortuous 
and coiled while the veins are not [15]. Thus when the 
finger is bent, the veins become kinked, while the arteries 
merely uncoil. 

4. Fingernail-Bone Interaction Model 
4.1. Basic Mechanism 

As described in the previous section, and shown in 
Figure 8, the fingernail is connected to the bone of the 
distal phalanx by a matrix of strong fibers, especially 
around the perimeter of the nail, which prevent the nail 
from detaching from the bone, even under very high 
tension. However these fibers do not prevent the nail from 
being compressed against the bone. Touch forces and 
posture are maintained through tension in the flexor and 
extensor tendons, which are attached at the proximal end 
of the bone. The tendons are thus able to exert torque on 
the bone but do not directly affect tension in the tissue of 
the fingertip. 

The bone itself has a distinctive arrowhead shape with 
protuberances at both ends where the fibers are attached. 
The bone does not extend all the way to the hyponychium 



where the nail detaches from the skin. This geometry is 
important in determining the regions of the nail bed that 
are affected by various forces within the fingertip. The 
shape and position of the bone and its effect on the nail 
bed capillaries is evidenced by Figure 1 0. On the left, the 
finger is pressed down against a flat surface. The profile 
of the bone is visible as the region of red where the 
capillaries are protected from collapsing due to the 
pressure. On the right, the nail is pressed against a 
transparent surface. In this case the profile of the bone is 
visible as the region of white where the capillaries 
between the nail and bone are compressed and collapsed* 
which also confirms the inability of the nail fibers to 
support compression. 




Figure 10. Nail-bone interaction. 



The important characteristics of the basic bone-nail 
interaction model can be summarized as the following: 

• The bone (distal phalanx) has a distinctive arrowhead 
shape with protuberances at both ends. 

• The fiber matrices between nail and bone support 
tension but not compression. 

• Tension in the tissue allows blood volume to increase 
while compression causes blood volume to decrease 

This basic model is now ready to be applied to explain 
the mechanism behind each of the various types of touch 
force and posture. 

4.2. Normal Force 

Figure ll depicts the nail-bone interaction and its 
hemodynamic effect for normal force. The top diagrams 
depict the ordinary state of the fingertip when no force is 
applied, The tissue is shaded lightly to indicate it is 
neither in tension nor compression. The transverse and 
dorsal views also depict the primary arteries (over the 
bone) and veins (beside the bone). The bottom diagrams 
show a z-force exerted beneath the fingertip and its 
corresponding reaction force exerted by the bone, which is 
achieved by tension in the flexor tendon. These forces 
compress the tissue of the pulp between the bone and the 
surface, as depicted by the area in white. Note that the 
compression extends to the area around the bone as well 
as beneath it This is because the nail bed fibers are in 
tension and pull the nail down with the bone, compressing 
all the tissue that is around the bone but beneath the nail. 



This includes the lateral aspects of the finger where the 
primary veins lie. Thus the veins are collapsed, causing 
blood to pool up in the protected capillaries between the 
nail and the bone, as depicted by the area shaded darkly. 



Nominal saggitalview transverse view dorsal view 

(section A-A) (section B-B) Ar-i 




Figure 11. Normal touch mechanism. 



4.3. Extension/Flexion 



Extension 




Figure 12. Bending mechanism. 



Figure 12 depicts the nail-bone interaction and its 
hemodynamic effect for finger extension and flexion. 
During extension* the extensor tendon pulls the bone 
upward against the fingernail, which is held in place by a 
reaction force from the proximal nail fold. Since the fibers 
of the nail bed do not support compression, the capillaries 
are collapsed between the bone and the nail, especially 
above the bony protuberance at the distal end of the bone. 

When the finger is flexed, the flexor tendon pulls the 
bone down away from the nail, relieving the compression 
between the nail and bone. Since the nail bed fibers are 
now in tension, the fingernail maintains a normal position 
relative to the bone. However, additional reddening 
occurs throughout the fingertip since flexion of the finger 
kinks the veins, causing blood to pool up in the capillaries 
throughout the fingertip. 



4 A Lateral Shear Force 

In the case of lateral shear force, the bone-nail 
interaction is more complex, as depicted in Figure 13. In 
order to apply shear, a normal force must be 
simultaneously exerted in order to allow for friction. It has 
already been established that when normal force is 
applied, the area around the bone is compressed and 
whitened. When lateral force is exerted in addition to 
normal force, a lateral reaction force is maintained in the 
bone by the ligaments of the joint, and the tissue of the 
pulp between the bone and surface experiences a shear 
force that pulls the tissue toward the far side of the shear 
vector. Thus the tissue at the far side becomes bunched up 
around the nail due to shear and whitened due to 
compression by the normal force. However, on the near 
side of the shear vector, the tissue is pulled away from the 
nail by the shear force. The tension of this pulling action 
prevents compression of the tissue at the near side, 
resulting in reddening. Furthermore, the tension at the 
near side putts the naif down on top of the bone there 
resulting in whitening. 



B-B 



A-A 




Figure 13. Lateral shear mechanism. 

4.5. Longitudinal Shear Force 

UnKke lateral shear force, the mechanism of 
longitudinal shear force is different for positive and 
negative forces, as depicted in Figure 14. When force is 
applied in the positive direction, as depicted in the bottom 
diagrams, the mechanism is similar to lateral shear. 
Applied shear and reaction force in the bone pulls the 
tissue proximally. At the distal end of the nail, the tissue is 
pulled away, generating tension and preventing the 
capillaries from collapsing, resulting in reddening. 
However the tension pulls the nail down on top of the 
bone, generating a whitening zone between the nail and 
bone. When force is applied in the negative direction* as 
depicted in the top diagrams, the tissue is bunched up at 
the distal end of the nail due to the shear and compressed 
due to normal force. 





Figure 14. Longitudinal shear mechanism. 
5. Verification and Applications 
5.1. Experimental Verification 

In order to verify that the blood Volume patterns predicted 
by the nail-bone interaction model are accurate, 
experimental data is collected using the fingernail sensor 
developed in previous work [10]. Figure 15 shows the 
arrangement of the optical components of the fingernail 
sensor. There are eight photodiodes in a two-dimensional 
array with six LEDs m between. The signal from each 
photodiode is dominated by the local blood volume 
beneath the fingernail 




Photodetectors 
LEDs 



LED and Photodiode Arrays 
Figure 15. Sensor arrangement. 

Figure 16 to Figure 19 show the responses of the eight 
photodiodes to normal force, bending, lateral shear force, 
and longitudinal shear force. In each figure, the eight plots 
are arranged in the same formation as the photodiodes on 
the fingernail. In each case, the human subject was asked 
to slowly vary the stimuli of interest over several cycles 
while holding the others constant The photodiode signals 
are plotted vs. the applied stimuli. The forces are 
measured using a three-axis force sensor and the joint 
angle is measured using a video camera. The hysteresis in 



each of the curves is due in part to the fact that the other 
three stimuli are not held perfectly constant while the 
stimuli of interest is being varied. 




Figure 16. Sensor response to normal force. 
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Figure 17. Sensor response to extension/flexion. 

When the magnitude of normal force increases, the 
photodiode signals increase, particularly towards the front 
and sides of the fingernail. This is because the whitening 
causes more light to be reflected, increasing the 
photodiode response. The front-most photodiode Levels 
off first because it is the first to go completely white. As 
more force is applied, the white zone grows proximal ly 
and the middle photodiodes continue to increase. 

When the finger is flexed (increasing joint angle), the 
middle of the nail is reddened and the photodiode signals 
decrease, particularly in the middle of the nail. For 
extension, the nail whitens and the signals increase. 

When lateral shear is applied, the photodiodes now act 
in a laterally asymmetric pattern as expected. For positive 



lateral shear (i.e. shear applied from right to left), the 
distal right side reddens (decreasing signal) and the distal 
left side whitens (increasing signal), just as predicted by 
the model. Likewise, for negative lateral shear (left to 
right), the distal left side reddens and the distal right side 
whitens. The photodiode at the front middle stays 
whitened, while the photodiodes in the rear fluctuate as 
the proximal whitening zone shifts around over the bone. 
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Figure 18. Sensor response to lateral shear. 




Figure 19. Sensor response to long, shear. 

When longitudinal shear is applied in the positive 
direction (front to back), the photodiodes signals all 
increase due to the broad whitening zone in the middle of 
the nail. However, when longitudinal shear is applied in 
the negative direction (back to front), the photodiode 
signals tend to remain the same or increase only slightly. 
Thus negative longitudinal shear will be difficult to 
distinguish based on sensor readings. 




5.2. Impact on Applications of Fingernail Sensor 

Since the fingernail sensor is capable of measuring 
shear force in addition to normal force, a potential 
application is to use the fingernail sensor as a wearable 
computer mouse, as shown in Figure 20. This is 
especially feasible since the sensors are most sensitive in 
the range of 0 to 2 N, which is comfortable for a human to 
apply on a continuous basis. 

However, because both the model and experimental 
evidence suggest that negative longitudinal shear may be 
difficult to detect, the wearable mouse application should 
be designed to function based only cm normal force, 
lateral shear, and positive longitudinal shear. It is 
possible that the two axes of cursor motion could be 
controlled by lateral shear and some combination of 
normal force and longitudinal shear. If the human presses 
the finger against the side of the computer monitor, then 
predicted normal force could be intuitively used to control 
horizontal cursor position, while predicted lateral shear 
force could be used to control vertical cursor position. 
Clicking would be achieved by dynamic tapping or by 
using a second finger. 




Figure 20. Wearable mouse application. 
6, Conclusions 

In conclusion, this paper developed a unified 
qualitative model that explains the fingernail-bone 
interaction and resulting blood volume patterns that occur 
in the fingernail bed when various forces and changes in 
posture are applied to the fingertip, This model is useful 
in understanding the measurements obtained using the 
fingernail sensor and designing sensor applications. 
Using the model, the sensor could be redesigned with an 
optimal configuration of photodiodes for distinguishing 
the various patterns of blood volume resulting from 
normal force, finger bending, lateral shear force, and 
longitudinal shear force. The understanding of the nail- 
bone interaction should also prove to be useful for 
understanding human grasping and manipulation. In 




future work, a quantitative model of the nail-bone 
interaction will be developed. 
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Abstract 

A new method for measuring both the posture of human 
fingers and shear force at human fingertips is presented. 
Instead of using a traditional electronic glove with 
bending sensors embedded along the finger and shear 
sensors embedded beneath the fingertip, a wearable 
Fingernail sensor is used to measure resulting changes in 
coloration of the fingernail. Since the sensor is mounted 
on the fingernail, finger posture can be measured without 
wearing a glove that hampers the motion of the fingers. 
Similarly, shear forces can be measured without covering 
the finger pad and obstructing the human's natural haptic 
sense. In the past, fingernail sensors with a one- 
dimensional array of photodetectors have been used to 
measure normal touching forces at the fingertip. A new 
fingernail sensor with a two-dimensional spatial array of 
photodetectors is constructed in order to measure finger 
posture and shear forces. Experiments are performed in 
order to measure the outputs of the photodetectors in 
response to changes in finger posture and applied normal 
and shear forces. Analysis is then performed to correlate 
the outputs to the inputs and provide a means of 
estimating normal forces, shear forces, and changes in 
finger posture. 

1. Introduction 

There is an increasing need for measuring both finger 
posture and forces acting between human hands and the 
environment. Finger posture is usually measured by 
wearing electronic gloves, which have been increasingly 
used in the robotics and virtual reality communities for a 
variety of human-machine interactions [1]. External 
finger forces are traditionally measured by placing force- 
sensing pads at the fingertips. A wide variety of such 
pads have been developed in the past for applications in 
robotics and medicine [2], using resistive, capacitive, 
piezoelectric, or optical elements to detect force [3], [4], 
[5]. These pads have often been placed in electronic 
gloves in order to monitor human skills and behavior 
during manipulation, perform teleoperation, and interact 
with machines and virtual environments [6], [7], [8], [9], 
[10], [11]. 

Shear forces, or sliding forces in the plane of the 
contacting surface, play an important role in human 
manipulation of objects. Shear forces, however, are very 



difficult to measure without interfering with the human's 
haptic sense. For robotic hands, several clever designs of 
shear force sensors have been presented. Howe and 
Cutkosky developed a superior shear force tactile sensor 
for detecting slip and surface texture [12]. Novak 
proposed a shear force sensor design using a capacitive 
sensor array [13]. Unfortunately these shear force sensors 
must be placed between the fingertip and the 
environment, and are therefore inappropriate for 
measuring human finger forces. A critical problem with 
traditional electronic gloves and force sensors in general 
is that they cover the fingers, blocking the natural human 
haptic sense as well as restricting the natural bending 
motion of the fingers. 

In a previous paper by the authors [ 14], a new approach to 
the detection of finger forces was presented to completely 
eliminate any impediment to the natural haptic sense. 
Namely, the finger force is not measured by placing a 
sensor pad between the finger skin and the environment 
surface, but is instead detected by an optical sensor 
mounted on the fingernail. This allows the human to 
touch the environment with bare fingers and perform fine, 
delicate tasks using the full range of haptic sense. 
Miniaturized optical components and circuitry allow the 
sensor to be disguised as a decorative fingernail covering, 
A lumped-parameter hemodynamic model was created in 
order to explain the color change at two locations under 
the fingernail. 

In this paper, a new fingernail sensor is constructed in 
order to allow for measurement of finger posture and 
shear forces in addition to normal touching forces. This 
new fingernail sensor has a two-dimensional spatial array 
of photodetectors for measuring shear forces along two 
different axes. Firstly, this paper describes the various 
color changes that result from touching, bending, and 
shear forces. An experimental platform is then assembled 
in order to measure the response of the fingernail sensor 
to known finger postures and shear forces. 
Experimentation and analysis are performed in order to 
relate the outputs of the fingernail sensor to the finger 
posture and shear force inputs. Based on the analysis, a 
method for simultaneously estimating normal forces, 
shear forces, and changes in finger posture is presented. 
Applications to a human-machine interface and a 
computer pointer are discussed. 



2. Principle 

2.1 Color Change Due to Touching 

As the human fingertip is pressed down on a surface with 
increasing force, the blood flow through the fingertip is 
affected, and a sequence of color changes is observed 
through the fingernail. In fact, the color change is 
characteristically non-uniform across the nail, resulting in 
distinct patterns of color change. These color patterns can 
be measured by placing arrays of light emitting diodes 
and photddetectors on the nail as shown in Figure 1 . The 
output of the photodetectors is proportional to the 
intensity of reflected light, which depends on the volume 
of blood in the nail bed. 

Capillary Hood 




Figure 1 : Sensing Principle (picture adapted from [151) 

Figure 2 shows the typical sequence of noticeable color 
changes with increasing force. As the touch force is first 
increased, the veins in the fingertip are collapsed, causing 
blood to pool up in the capillaries beneath the nail, 
resulting in the reddening effect. As the force continues 
to increase, the force propagates around the bone, 
collapsing the capillaries at the tip of the nail bed, 
resulting in a white zone at the tip of the nail. 
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Figure 2: Fingernail Color Change Due to Touching 

A variety of descriptions of the anatomy and physiology 
of the fingertip can be found in medical literature [16], 
[17], [18], [19], [20]. In previous work [15], the 
mechanism of these color changes was discussed in more 
detail and an anatomically based dynamic model was 
created in order to explain the underlying changes in 
blood volume. However, normal touch force is not the 
only action that results in a change in fingernail color. 
Alterations in finger posture as well as application of 
shear force result in different color patterns than those 
shown above. 



2.2 Color Change Due to Bending/Extensi6n 

When the posture of the finger is altered, i.e. the joints of 
the finger are bent or extended, the color of the finger 
changes as shown in Figure 3. When the finger is 
extended, a tension is set up in the tissues of the nail bed 
that collapses the capillaries. When the finger is bent, that 
tension is relieved and the capillaries fill with blood 
again. Creasing of the veins during bending may also 
contribute to blood pooling up in the capillaries. 
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Figure 3: Fingernail Color Change Due to Bending 

The color changes shown above are concentrated near the 
center of the nail, whereas the color changes due to 
normal touching occur particularly towards the tip of the 
nail. Therefore it should be possible to distinguish 
between a touching action and a change in finger posture 
based on observable changes in fingernail color patterns. 

23 Color Change Due to Shear 

When shear forces are applied to the palmar surface of the 
fingertip, yet another pattern of color changes result, as 
shown in Figure 4. If the shear force is applied 
longitudinally* a tension in the tissues of the nail bed is set 
up, resulting in a broad-whitening effect over the front of 
the nail. If the direction of the shear force deviates in 
either direction from the longitudinal axis, then the white 
zone shifts towards the right or left of the nail as shown. 
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Figure 4: Fingernail Color Change Due to Shear 

Unlike the color change due to bending/extension, the 
white zone due to shear is concentrated toward the front 
of the nail. However the white zone for shear extends 
much farther back than in the case of normal touching 
force, Therefore it should be possible to distinguish shear 
forces from bending/extension as well as from normal 
touching. 





3. Experimental Apparatus and Method 

3.1 Fingernail Sensor Apparatus 

In order to recognize normal touch force, 
bending/extension, and shear forces (both longitudinal 
and lateral), it is necessary to measure the color of the 
fingernail using a 2-dimensionaI spatial array of 
photodetectors. Figure 5 shows a picture of the fingernail 
sensor that is used for experimentation. The first picture 
shows the bottom side of the sensor, where the 2-D array 
of photodiodes is mounted towards the end of the sensor. 
Four photodiodes run along the longitudinal axis of the 
nail with two additional photodiodes on either side for a 
total of eight photodiodes. Six light emitting diodes are 
distributed evenly between the photodiodes in order to 
illuminate the entire nail bed with infrared light at the 
isobestic wavelength (770nm). The second picture shows 
the top of the nail sensor, where the photodiode signals 
are amplified and then sent to an analog-to-digital 
converter. 




Figure 5: Fingernail Sensor with 2-D Array of 
Photodetectors 



After the optical and electrical components are in place as 
shown in Figure 5, the bottom side is molded to the shape 
of the human fingernail using transparent epoxy. The top 
is covered with opaque epoxy to shield the sensor from 
ambient lighting. The sensor is then attached to the 
fingernail (as seen in Figures 6 and 7) using special 
transparent double-sided adhesive tabs. Thin wires then 
connect the signals to the wrist and then to the A/D 
converter and power supply. 

3.2 Posture Measurement 

In order to experiment with finger posture, it is necessary 
to be able to measure the angles of the joints of the finger 
while measurements are simultaneously taken from the 
fingernail sensor. The easiest method would be to wear 
an electronic glove; unfortunately sueh gloves constrict 
the fingers and interfere with the adhesion of the sensor to 
the nail. Therefore the fingers are left free and a video 
tracking system is used instead, as shown in Figure 6. A 
pair of colored markers is placed on each joint of the 
finger and video is taken continuously during the 
experiment. An image processing algorithm is then used 
off-line to track the centroids of the markers and compute 
the angle of each joint for each time index. The clock on 
the computer rnonitc-r is used to Synchronize the time 
index of the joint angle measurements to that of the 
photodetector measurements. 



Figure 6: Completed Fingernail Sensor and Video 
Tracking System for Posture Measurement 

3.3 Force Measurement 

In order to experiment with shear force, it is necessary to 
be able to measure the shear forces that are applied to the 
finger along both the X-axis and Y-axis as well as the 
normal touching force along the Z-axis. Figure 7 shows a 
picture of the force measurement system that is used. A 
Nanol7 6-axis force/torque transducer from ATI 
Automation is used in order to measure forces in the X, 
Y, and Z directions. The shear forces F x and Fy have a 
sensing range of ± 12 N and a resolution of 0.8xl0" 3 N. 
F z has a range of ± 17 N and a resolution of 1.6x1 0" 3 N. 
A rubber pad is placed between the finger and the sensor 
in order to maximize the coefficient of friction and thus 
decrease the dependence of the shear force on the normal 
force. 




Figure 7 : Force Measurement Platform 

Ideally, the experiments would be designed so that the 
finger joints would be held in place while an automated 
apparatus applies precise shear forces. Unfortunately, a 
device that would hold the finger in place would put 
pressure on the finger and affect the blood flow, thus 
affecting the nail color. Therefore, the human must be 
relied upon instead to apply the shear forces as precisely 
as possible. A visual feedback system consisting of 
gauges on a computer monitor is used to display to the 
human in real-time the forces that are applied as the finger 
is pressed on the platform. 



4. Experimental Results 



4.1 Results for Posture Measurement 

Figures 8 through 10 show the results of experiments for 
finger posture measurement. Starting from a fully bent 
position, the finger is first fully extended and then fully 
bent once again. The responses of each of the eight 
photodetectors are shown in Figure 8. The plots are 
arranged in the figure to reflect the relative positions of 
the photodetectors on the fingernail. As shown by the 
data, the photodetector outputs all respond in phase "with 
the bending. 

Photodector outputs are all in phase 
with bending 
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Figure 8: Response to Finger Bending 
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Figure 10: Sensor Output vs. Bend Angle 

Using the video tracking system shown in Figure 6, the 
joint angles were computed from video frames spanning 
the bending range of the finger. The finger consists of 
three joints, not all of which can be flexed independently. 



Figure 9 shows the correlation between the joints using 
data from the experiment. Since the knuckle joint can be 
flexed independently of the other two and does not appear 
to influence the photodetector outputs, the knuckle angle 
Ji is held constant for this experiment. The middle and 
distal joints, h and J 3 , cannot be flexed independently and 
have a roughly linear dependence, as shown in Figure 9. 
Therefore, the photodetector outputs can be treated simply 
as a function of J 2 , and are plotted as such in Figure 10, 
showing a characteristic pattern of sensor outputs for 
bending. 

4.2 Results for Force Measurement 
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Figure 11: Response to Normal Force 




Figure 12: Sensor Output vs. Normal Force 

Figures 11 and 12 show the results of experiments for 
measurement of normal force, F z . The finger is slowly 
pressed down on the measurement platform with 
increasing force and then slowly lifted up again for 
several cycles. The responses of the photodetectors are 
plotted vs. time in Figure II. The normal force F z is 
measured by the 6-axis force sensor and is shown at the 
bottom of Figure 11. In Figure 12, the photodetector 
outputs are plotted as a function of F z , showing a 
characteristic pattern of sensor outputs for normal force. 



This pattern is distinguishable from that of bending by the 
differences in curvatures and slopes. 

Figures 13 and 16 show the results of the experiments for 
shear force measurement. First, the human applies a 
periodic lateral shear force (magnitude~3 N, period~0.5 
sec) to the palmar surface of the fingertip. The responses 
of the photodetectors are shown in Figure 1 3 along with 
the applied shear force. In this case, unlike the cases of 
finger bending and normal force, the photodetectors do 
not all respond in phase. The photodetectors on the left 
side of the nail react in phase with the shear force while 
the photodetectors on the right side of the nail react 180 
degrees out of phase with the shear force. In Figure 14, 
the photodetector outputs are plotted as functions of the 
applied lateral shear force, F x , showing a characteristic 
pattern of sensor outputs for lateral shear. The opposite 
slopes for photodetectors 5 and 7 make this pattern easily 
distinguishable from the patterns for bending and normal 
force. 



photodetector outputs are plotted as functions of the 
applied longitudinal force, F Y , showing a characteristic 
pattern of sensor outputs for longitudinal shear. The non- 
linear v-shaped curves make this pattern easily 
distinguishable from bending, normal force, and lateral 
shear force. 





Figure 15: Response to Longitudinal Shear 
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Figure 13: Response to Lateral Shear 




Figure 14: Sensor Output vs. Lateral Shear 

Next, the human applies a periodic longitudinal shear 
force (magnitude^ N, period~0.5 sec) to the palmar 
surface of the fingertip. The responses of the 
photodetectors are shown in Figure 15 along with the 
applied shear force. In this case the responses all seem to 
be in phase, but their behavior is much more non-linear. 
This is more evident in Figure 1 6, where the 




Figure 16: Sensor Output vs. Longitudinal Shear 



5. Analysis 
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Figure 17: Predicted vs. Actual Forces/Posture 



As seen in section 4, each of the four inputs under 
consideration results in a unique pattern of photodetector 
responses. It should therefore be possible to 
simultaneously estimate the inputs based on the 
photodetector outputs. As a first step, a multivariate 
linear least squares regression is performed using data 
from the four experiments to create a linear model: 

F = P(P-P 9 ) + £, 

where P is an 8x1 vector of photodetector outputs, and F 
is a 4x1 vector consisting of F x , F y , F z , and J 2 , 0 is a 4x8 
matrix relating the two, and e is the residual vector. The 
photodetector outputs from the four experiments can then 
be used to generate predicted values of the three forces 
and bending angle. The results of the regression are 
shown in Figure 17, where the predicted inputs are plotted 
vs. the measured inputs for each of the four experiments. 
The diagonal line passing through each plot represents the 
ideal one-to-one relationship between actual and predicted 
inputs. This linear model appears to work well over the 
dynamic ranges of F x and F z , but not so well for F Y and 
J 2 , whose predictions are not as centered about the 
diagonal. This problem must be solved in future work by 
using a non-linear model to relate inputs and outputs. 

6. Conclusion 

In this paper, a new method for measuring the posture of 
human fingers and shear forces at the fingertips has been 
presented. A wearable fingernail sensor with a 2- 
dimensional array of photodetectors is capable of 
measuring the pattern of color change (or change in blood 
volume) of the fingernail. Experiments were performed 
to demonstrate the relationships between the 
photodetector outputs and various inputs including finger 
joint angle, normal force, lateral shear, and longitudinal 
shear force. The results show that each input has a unique 
pattern of response. A multivariate linear regression was 
used to create a model capable of predicting all four 
inputs based on the photodetector outputs. Future work 
will involve more extensive experimentation and design 
of non-linear models that are capable of predicting finger 
forces and posture with greater accuracy. 

This method will be useful for monitoring human skills 
and behavior during manipulation, performing 
teleoperation, and interacting with machines and virtual 
environments. By measuring shear forces at the human 
fingertip without placing a sensor underneath the 
fingertip, it would even be possible to replace a computer 
mouse with a wearable fingernail sensor. The user could 
simply push his or her finger against a surface to generate 
shear forces in two dimensions. Incorporating normal 
touching force and finger bending could further augment 
the functionality of such an interface. 
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